A head-mounted display (HMD) or a like device produces a modified-reality environment by combining virtual objects and representations of real-world objects. The HMD will deliver poor user experience whenever it places the virtual objects at erroneous or otherwise non-ideal locations in the modified-reality environment. Indeed, for some modified-reality applications, the errant placement of virtual objects can significantly degrade the utility of the HMD, and may even lead to safety-related concerns.